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A method for filtering digital images, and a filtering device 

The present invention relates to a method for removing visual artefacts 
from a frame of a digital image, which has been coded by blocks and 
5 then decoded, in which method at least one pixel from at least one side 
of a block boundary is selected for filtering, filtering Is performed on the 
block boundary between two adjacent blocks, and the blocks are 
scanned in a certain order. The present invention also relates to a 
device for removing visual artefacts from a frame of a digital image, 

10 which has been coded by blocks and then decoded, which device com- 
prises means for selecting at least one pixel from at least one side of a 
block boundary for filtering, means for performing filtering on a block 
boundary between two adjacent blocks, and means for scanning the 
blocks in a certain order. The present invention also relates to an 

15 encoder, decoder, and codec comprising means for coding and decod- 
ing a digital image by blocks, which encoder comprises means for 
selecting at least one pixel from at least one side of a block boundary 
for filtering, means for performing filtering on a block boundary between 
two adjacent blocks, and means for scanning the blocks in a certain 

20 order. The present invention relates furthermore to a mobile terminal 
comprising a video codec, which comprises means for coding and 
decoding a digital image by blocks, which codec comprises means for 
selecting at least one pixel from at least one side of a block boundary 
for filtering, means for performing filtering on a block boundary between 

25 two adjacent blocks, and means for scanning the blocks in a certain 
order. The present invention relates furthermore to a storage medium 
for storing a software program comprising machine executable steps for 
coding and decoding a digital video signal by blocks, for selecting at 
least one pixel from at least one side of a block boundary for filtering, 

30 for performing filtering on a block boundary between two adjacent 
blocks, and for scanning the blocks in a certain order. 

An arrangement like the one shown in Figure 1 is generally used for 
transferring a digital video sequence in compressed form. The digital 
35 video sequence is formed of sequential frames. In some prior art digital 
yideo transmission systems, for example ITU-T H.261/H.263 
recommendations, three frame types are defined: an l-frame (Intra), a 
P-frame (Predicted or Inter), and a B-frame (Bi-directional). The l-frame 
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is generated solely on the basis of information contained in the image 
itself, wherein at the receiving end, this l-frame can be used to form the 
entire image. P-frames are formed on the basis of a preceding l-frame 
or P-frame, wherein at the receiving stage the preceding l-frame or P- 
5 frame is correspondingly used together with the received P-frame in 
order to reconstruct the image. In the composition of P-frames ( for 
instance motion compensation is used to compress the quantity of 
information. B-frames are formed on the basis of one or more 
preceding P-frames or l-frames and/or one or more following P- or I- 
10 frames. 

The frames are further divided into blocks. One frame can comprise 
different types of blocks. A predicted frame (e.g. Inter frame) may also 
contain blocks that are not predicted. In other words, some blocks of a 

15 P-frame may in fact be intra coded. Furthermore, some video coders 
may use the concept of independent segment decoding in which case 
several blocks are grouped together to form segments that are then 
coded independently from each other. All the blocks within a certain 
segment are of the same type. For example, if a P-frame is composed 

20 mainly of predicted blocks and some Intra-coded blocks, the frame can 
be considered to comprise at least one segment of intra blocks and at 
least one segment of predicted blocks. 
I 

In tijie YUV colour model, the image is represented by a luminance 
25 component and two chrominance components. The luminance informa- 
tion! in tne ima 9e typically is transformed with full spatial resolution. 
Both chrominance signals are spatially subsampled, for example a field 
of 16 x 16 pixels is subsampled into a field of 8x8 pixels. The differ- 
ences in the block sizes are primarily due to the fact that the eye does 
30 not (discern changes in chrominance equally well as changes in lumi- 
nance, wherein a field of 2 x 2 pixels is encoded with the same chromi- 
nance value. 

J 

Typjcally, image blocks are grouped together to form macroblocks of 
35 2x2jblocks. The macroblock contains usually 16 pixels by 16 rows of 
luminance samples, the mode information, and possible motion vectors. 
The; macroblock is divided into four 8x8 luminance blocks and to two 8 
x 8 chrominance blocks. Scanning (and encoding/decoding) proceeds 
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macroblock by macroblock, conventionally from the top-left to the 
bottom-right corner of the frame. Inside one macroblock the scanning 
(and encoding/decoding) order is from the top-left to the bottom-right 
corner of the macroblock. 

5 

Referring to figure 1 , which illustrates a typical encoding and decoding 
system (codec) used, for example, in the transmission of digital video, a 
current video frame to be coded comes to the transmission system 10 
as input data l n (x,y). In the differential summer 11 it is transformed into 

10 a prediction error frame E n (x,y) by subtracting from it a prediction frame 
Pn(x,y) formed on the basis of a previous image. The prediction error 
frame is coded in block 12 in the manner described hereinafter, and the 
coded prediction error frame is directed to a multiplexer 13. To form a 
new reconstructed frame, the coded prediction error frame is also 

15 ■ directed to a decoder 14, which produces a decoded prediction error 
frame E n (x,y) which is summed in a summer 15 with the prediction 
frame P n (x,y), resulting in a reconstructed frame l n (x,y). The 
reconstructed frame is saved in a frame memory 16. To code the next 
frame, the reconstructed frame saved in the frame memory is read as a 

20 reference frame R n (x,y) and is transformed into a new prediction frame 
Pn(x,y) in a motion compensation and prediction block 1 7 according to 
the formula 

P n (x,y) = R n [x + Dx(x,y), y + Dy(x,y)] (1 ) 

The pair of numbers [Dx(x,y), Dy(x,y)] is called the motion vector of the 
25 pixel at location (x.y) and the numbers Dx(x,y) and Dy(x.y) are the hori- 
zontal and vertical shifts of the pixel. They are calculated in a motion 
estimation block 18. The set of motion vectors [Dx( ), Dy(-)] consisting 
of all motion vectors related to the pixels of the frame to be compressed 
is also coded using a motion model comprising basis functions and 
30 coefficients. The basis functions are known to both the encoder and the 
decoder. The coefficient values are coded and directed to the mul- 
tiplexer 13, which multiplexes them into the same data stream with the 
coded prediction error frame for sending to a receiver. In this way the 
amount of information to be transmitted is dramatically reduced. 

35 
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Some frames can be partly, or entirely, so difficult to predict using only 
the reference frame R n (x.y) that it is not practical to use motion 
compensated prediction when coding them. These frames or parts of 
frames are coded using intracoding without any prediction from the 
5 reference frame R n (x,y), and accordingly motion vector information 
relating to them is not sent to the receiver. In prior art, for l-frames or 
those parts of P-frames which are intra-coded, another kind of 
prediction may be employed, namely intra prediction. In this case, the 
reference is formed by the previously decoded and reconstructed 
10 blocks which are part of the same frame (or slice if independent 
segment decoding is used). 

In the receiver 20, a demultiplexer 21 separates the coded prediction 
error frames and the motion information transmitted by the motion 

15 vectors and directs the coded prediction error frames to a decoder 22, 
which produces a decoded prediction error frame E n (x,y), which is 
summed in a summer 23 with the prediction frame P n (x,y) formed on 
the basis of a previous frame, resulting in a decoded and reconstructed 
frame T n (x,y). The decoded frame is directed to an output 24 of the 

20 decoder and at the same time saved in a frame memory 25. When 
decoding the next frame, the frame saved in the frame memory is read 
as a reference frame R n (x,y) and transformed into a new prediction 
frame in the motion compensation and prediction block 26, according to 
formula (1) presented above. 

25 

The coding method used in the coding of prediction error frames and in 
the intracoding of a frame or part of a P-frame to be sent without using 
motion prediction, is generally based on a transformation, the most 
common of which is Discrete Cosine Transformation, DCT. The frame 

30 is divided into adjacent blocks sized e.g. 8x8 pixels. The 
transformation is calculated for the block to be coded, resulting in a 
series of terms. The coefficients of these terms are quantized on a dis- 
crete scale in order that they can be processed digitally. Quantization 
causes rounding errors, which can become visible in an image recon- 

35 structed from blocks, so that there is a discontinuity of pixel values at 
the boundary between adjacent blocks. Because a certain decoded 
frame is used to calculate the prediction frame for subsequent 
predicted (P) frames, these errors can be propagated in sequential 
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frames, thus causing visible edges in the image reproduced by the 
receiver. Image errors of this type are called blocking artefacts. 
Furthermore, if intra-prediction is used, blocking artefacts may also 
propagate from block to block within a given frame. In this case 
5 blocking artefacts typically lead to visual effects which are specific to 
the type of intra prediction used. It should therefore be appreciated that 
there exists a significant technical problem relating to the spatial and 
temporal propagation of blocking artefacts in digital images that are 
coded for transmission and subsequently decoded. 

10 

The principles presented above are also applicable to a situation where 
segmented frames are used. In that case the coding and decoding is 
performed in segments of the frame, according to the type of blocks in 
each segment. 

15 

Some prior art methods are known for removing blocking artefacts. 
These methods are characterized by the following features: 

- determining which pixels require value correction in order to remove a 
20 blocking artefact, 

- determining a suitable low-pass filtering for each pixel to be corrected, 
based on the values of other pixels contained by a filtering window 

. placed around the pixel, 

..:•* 25 

- calculating a new value for the pixel to be corrected, and 

- rounding the new value to the closest digitized pixel value. 

* * * 

30 Factors that influence the selection of a filter and the decision to use 
filtering can be, for example, the difference between the values of 
P' xels across the block boundary, the size of the quantization step of 
".' tne coefficients received as the transformation result, and the differ- 

er, ce of the pixel values on different sides of the pixel being processed 

35 

• • • 

In P"'o r art methods the filtering of blocking and other types of visual 
artefacts is performed frame by frame, i.e. the whole frame is first 
"• decoded and then filtered. As a result, the effects of blocking artefacts 
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35 



easily propagate within a frame or from one frame to the next when 
prior art methods are used. This is especially true when predictive 
intracoding is used. 

It has been found that prior art methods also tend to remove lines that 
belong to real features of the image. On the other hand, prior art meth- 
ods are not always capable of removing all blocking or blocking related 
artefacts. 

A primary objective of the method according to the invention is to limit 
the propagation of blocking artefacts within frames and from one frame 
to the next. Another objective of the present invention is to present a 
new kind of filtering arrangement for removing blocking and other 
blocking related artefacts which are especially visible when predictive 
15 intracoding is used. The invention also has the objective that the 
method and SLSSOciated device operate more reliably and efficiently 
than prior art solutions. 



* * * 



The objectives of the invention are achieved by performing block 
boundary filtering in a particular order and substantially immediately 
after an image block is decoded and there is at least one block 
boundary avaijable to be filtered. Among other things, this provides the 
advantage that the spatial and temporal propagation of blocking and 
other visual artefacts is limited to a greater degree than in prior art 
methods. Furthermore, the results of previous block boundary filtering 
operations can be utilised in the coding, decoding and filtering of 
subsequent blpcks. In other words, pixel values modified/corrected in 
connection with the filtering of one boundary are then available for use 
when coding, decoding and filtering other block boundaries. 

The method according to the invention for removing blocking and other 
visual artefacts (which are mainly caused by blocking artefacts) from a 
frame that has been coded by blocks, is characterized in that filtering 
on a block boundary is performed substantially immediately after the 
block is decoded, and that the order of filtering the boundaries is 
predetermined. 



24/01 00 lfl:05 FAX 0.1 2886262 



TAMPEREEN PATENT 



12)000 



7 



The invention also relates to a device for implementing the method 
according to the invention. The device according to the invention is 
characterized in that filtering on a block boundary is performed 
substantially Immediately after the block is decoded, and that the order 
5 of filtering the boundaries is predetermined. 

The Invention also relates to an encoder for encoding digital images 
that implements the method of the invention. The encoder according to 
the invention is characterized in that filtering on a block boundary is 
10 performed substantially immediately after the block is decoded, and 
that the order of filtering the boundaries is predetermined. 

The invention also relates to a decoder for decoding digital images that 
implements the method according to the invention. The decoder 
15 according to the invention is characterized in that filtering on a block 
boundary is performed substantially immediately after the block is 
decoded, and that the order of filtering the boundaries is 
predetermined. 

20 The invention also relates to a codec for encoding and decoding digital 
images that implements the method according to the invention. The 
codec according to the invention is characterized in that filtering on a 
block boundary has been performed substantially immediately after the 
>:> block is decoded, and that the order of filtering the boundaries is 

25 predetermined. 

i • » » t 

fat 

The invention also relates to a mobile terminal having digital image 
transmission capability and implementing the method according to the 
... invention. The mobile terminal according to the invention is 

30 characterized in that filtering on a block boundary is performed 
substantially immediately after the block is decoded, and that the order 
of filtering the boundaries is predetermined. 

-» 

: " : The invention also relates to a strorage medium storing a software 

": 35 program with instructions implementing the method according to the 
invention. The strorage medium according to the invention is 
" : . characterized in that the software program further comprises machine 

< executable steps for performing filtering on a block boundary 
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substantially immediately after the block is decoded, and that the order 
of filtering the boundaries is predetermined. 

Because blocking artefacts occur at block boundaries, the filtering 
5 method according to the invention is only applied to pixels at block 
boundaries and the immediate vicinity thereof. Edges that are part of 
the image can reside anywhere in the image area. In order that only 
pixels containing blocking artefacts are selected for corrective filtering 
and that the quality of edges that are part of the image is not affected 
10 during filtering, the following assumptions were made in the study that 
resulted in the invention: 

The changes in pixel value associated with edges that are part of the 
image, are generally larger than those associated with blocking 
1 5 artefacts, and 

Those edges within the image, where the pixel value change is small 
do not suffer considerably from the rounding of the pixel value differ- 
ence caused by filtering. 

20 

Because an image to be coded is generally divided into blocks both 
vertically and horizontally, the image contains both vertical and horizon- 
tal block boundaries. With regard to vertical block boundaries, there are 
pixels to the right and left of the boundary, and with regard to horizontal 
25 block boundaries, there are pixels above and below the boundary. In 
general, the location of the pixels can be described as being on a first 
or a second side of the block boundary. 

The method and associated device according to the invention signifi- 
30 cantly limits propagation of visual anomalities due to blocking artefacts 
from previously reconstructed blocks to subsequently reconstructed 
blocks within the same frame, or within the same segment, if independ- 
ent segment decoding is used. Propagation of blocking artefacts within 
frames and from one frame to the next is also reduced. By using the 
35 method and device according to the invention a larger number of 
blocking and blocking related artefacts can be removed without 
weakening the real image edges unreasonably. 
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In the following, the invention will be described in more detail with 
reference to the preferred embodiments and the accompanying draw- 
ings, in which 



5 Figure 1 represents a digital video codec according to prior art, 



10 



20 
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Figure 2 



represents an advantageous block scanning order in the 
method according to a preferred embodiment of the inven- 
tion, 



Figure 3 represents an advantageous block boundary filtering order 
after reconstruction of a biock according to a preferred 
embodiment of the invention, 

15 Figure 4 represents an advantageous filtering order in the preferred 
method according to the invention, 

Figure 5 represents a digital image block transfer system for 
implementing a method according to the invention, 



In the 

ments 



In a 
that 



Figure 6 represents a flow diagram of a method according to the 
invention, and 

Figure 7 is a schematic representation of a portable teleconferencing 
device implementing a method according to the invention. 



following description of the invention and its preferred embodi- 
5, reference will be made mostly to Figures 2 to 6. 



30 In the following, the operation of a digital image decoder is described. 



35 not 



digital image transfer system according to the invention, such as 
illustrated in figure 5, the block and block boundary scanning order 
usecjl in the encoder and decoder are the same, i.e. known to both 
encoder and decoder. However, the particular scanning order chosen is 



essential for implementation of the method according to the 
invention. Figure 2 shows an advantageous scanning order of a frame 
comprising groups of blocks, e.g. macroblocks. First, the top-left block 
B1 is decoded, i.e. pixel values representing e.g. the luminance 



00 19:06 FAX 03 2886262 



TAMPERE EN PATENT 



10 



©012 



information of the block are reconstructed and saved into a frame 
buffer. Then, the top-right block B2 is decoded and saved into the 
frame buffer. Now, the first vertical boundary R12 between the top-left 
block B1 and top-right block B2 has a decoded block at both sides, so 
5 the boundary R12 can be filtered. Advantageously only those pixel 
values which are changed by the filtering are updated in the frame 
buffer. There are many known methods for performing filtering on the 
block boundary. One prior art filtering method which can be 
implemented with the invention is disclosed in international patent 
10 publication WO 98/41 025 which is to be considered as a reference 
here. 

Next, the bottom-left block B3 of the group of blocks in question is 
decoded and saved into the frame buffer. Now, the first horizontal 

15 boundary R13 between the top-left block B1 and bottom-left block B3 
has a decoded block on both sides, wherein the first horizontal 
boundary R13 can also be filtered. In the filtering of the first vertical 
boundary R12 some pixel values near the first horizontal boundary R13 
may have changed. Advantageously, these modified values are used in 

20 the filtering of the first horizontal boundary R13. This helps to further 
restrict propagation of visual artefacts from previously reconstructed 
blocks to subsequent reconstructed blocks within the same frame, or 
same segment, if independent segment decoding is used. 

25 Now the fourth block B4 inside the macroblock is decoded and saved 
into the frame buffer. When decoding of the fourth block B4 is complete 
there exist two additional boundaries having a decoded block on either 
side: the second vertical boundary R34 and the second horizontal 
boundary R24. Therefore, both of said boundaries R34, R24 can now 

30 be filtered. In this advantageous embodiment of the method according 
to the invention the filtering is performed such that the second vertical 
boundary R34 is filtered first, the filtering result is saved in the frame 
buffer, and the second horizontal boundary R24 is filtered 
subsequently. In a general case, if two boundaries (e.g. to the left of 

35 and above) a current block are filtered then the changed pixel values 
resulting from the first filtered boundary are used when filtering the 
other boundary. 
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In an advantageous embodiment of the invention, for a certain block, 
filtering can be performed across one, two, three, four, or none of the 
boundaries of the block, depending on which scanning order is adopted 
for coding the blocks inside a frame, or segment. In the preferred mode 

5 of implementation the order in which blocks are reconstructed is 
illustrated in Figure 2. As depicted, four blocks are grouped together to 
form macroblocks of 2x2 blocks. Scanning then proceeds macroblock 
by macroblock from the top-left to the bottom-right corner of the frame. 
Inside one macroblock the scanning order is from the top-left to the 

10 bottom-right corner of the macroblock. Due to this particular scanning 
order, in the preferred mode of implementation a maximum of two 
boundaries (to the left of and/or above) a block become available for 
filtering, when a block is reconstructed. The order in which block 
boundaries are advantageously examined for filtering is illustrated in 

15 Figure 3 (left boundary first and then the upper one). The remaining 
boundaries, i.e. to the right and below, are filtered only when recon- 
struction of an adjacent block to the right and, respectively, below is 
completed. In the event that the block is located at the frame border or 
at a segment border, the corresponding block boundary/boundaries 

20 is/are not filtered since there is no adjacent block to filter across the 
common boundary. In the preferred mode of implementation the order 
in which the block boundaries are filtered inside one frame is illustrated 
in Figure 4 for a small frame size of 6x4 blocks. The numbers on the 
block boundaries represent the filtering order according to an advanta- 

25 geous embodiment of the present invention. In practical applications 
the frame typically comprises more than 6x4 blocks, but it is clear from 
the description above how the preferred filtering order can be extended 
to frames and segments which comprise more (or less) than 6x4 
blocks. 

30 

In general, the system described in this invention can be applied to 
block-based still image coding as well as to all kinds of coding in block- 
based video coders: I, P, B, coded and not-coded. The filtering 
described in this invention works for a frame that is divided into NxM 
35 blocks for coding. 

In the method according to the invention, filtering is applied only across 
those block boundaries that are adjacent to other previously 
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reconstructed blocks as soon as a block is reconstructed. If 
independent segment decoding is used, filtering is only applied across 
those block boundaries that are adjacent to previously reconstructed 
blocks belonging to the same segment as the current block. 

5 

An essential feature of the method according to the invention is the fact 
that the result of filtering is made available to the digital image coding 
system before reconstructing a subsequent block in the frame. This is 
especially advantageous for predictive intracoding where prediction of a 

10 block is performed with reference to previously reconstructed blocks 
within the same frame (or segment, if independent segment decoding is 
utilised). The current scheme is advantageous for any given block 
coding scheme which uses prediction of a block from previously 
reconstructed blocks inside the same frame or the same segment. In 

15 addition to the advantages achieved by prior art solutions, this way of 
operation prevents propagation of visual artefacts from previously 
reconstructed blocks to subsequently reconstructed blocks within the 
same frame, or the same segment, if independent segment decoding is 
used. Reconstruction of a certain block is therefore dependent on fil- 

20 tered data derived from previously reconstructed blocks. In order to 
avoid encoder-decoder mismatch, the decoder not only has to execute 
the same scheme of filtering but must also perform filtering operations 
in the same order as the encoder. The method also prevents 
propagation of blocking artefacts from one frame to the next, because 

25 blocking artefacts in a frame used e.g. in prediction are reduced by the 
method according to the invention. 

In the following, the transmission and reception of video frames in a 
video transmission system is described with reference to the digital 

30 image transfer system presented in Figure 5 and the flow diagram in 
Figure 6. Operation of the block boundary filtering method according to 
the invention will first be described in connection with the encoder of 
the transmission system, in the situation that a frame of a digital image 
sequence is encoded in Intra (l-frame) format and using some form of 

35 Intra block prediction, in which a block of the Intra frame may be coded 
with reference to other previously coded Intra blocks within the same 
frame. Filtering of block boundaries in the decoder of the transmission 
system will then be described for a corresponding Intra-coded frame 
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received for decoding at the receiver. Finally, application of the block 
boundary filtering method according to the invention to Inter-coded 
frames (P-f rames) will be described. 

5 Assuming that a frame is to be encoded in Intra format using some form 
of Intra prediction, encoding of the frame proceeds as follows. The 
blocks of the frame to be coded are directed one by one to the encoder 
50 of the video transfer system presented in Figure 5. The blocks of the 
frame are received from a digital image source, e.g. a camera or a 
1 0 video recorder (not shown) at an input 27 of the image transfer system. 
The frame can be stored temporarily in a frame memory (not shown), or 
alternatively, the encoder receives the input data directly block by 
block. 

15 The blocks are directed one by one to a prediction method selection 
block 35 that determines whether the pixel values of the current block 
to be encoded can be predicted on the basis of previously Intra-coded 
blocks within the same frame or segment. In order to do this, the 
prediction method selection block 35 receives input from a frame buffer 

20 of the encoder 33, which contains a record of previously encoded and 
subsequently decoded and reconstructed Intra blocks. In this way, the 
prediction method selection block can determine whether prediction of 
the current block can be performed on the basis of previously decoded 
and reconstructed blocks. Furthermore, if appropriate decoded blocks 

25 are available, the prediction method selection block 35 can select the 
most appropriate method for predicting the pixel values of the current 
block, if more than one such method may be chosen. It should be 
appreciated that in certain cases, prediction of the current block is not 
possible because appropriate blocks for use in prediction are not 

30 available in the frame buffer 33. It should also be appreciated that the 
prediction method selection block is an optional element in the coding 
system and is present in the situation that more than one Intra 
prediction method can be selected. In the situation where more than 
one prediction method is available, information about the chosen 

35 prediction method is supplied to multiplexer 13 for further transmission 
to the decoder. It should also be noted that in some prediction 
methods, certain parameters necessary to perform the prediction are 
transmitted to the decoder. This is, of course, dependent on the exact 
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implementation adopted and in no way limits the application of the block 
boundary filter according to the invention. 

Pixel values of the current block are predicted in the Intra prediction 
5 block 34. The Intra prediction block 34 receives input concerning the 
chosen prediction method from the prediction method selection block 
35 and information concerning the blocks available for use in prediction 
from frame buffer 33. On the basis of this information, the Intra 
prediction block 34 constructs a prediction for the current block. The 

10 predicted pixel values for the current block are sent to a differential 
summer 28 which produces a prediction error block by taking the 
difference between the pixel values of the predicted current block and 
the actual pixel values of the current block received from input 27. Next, 
the error information for the predicted block is coded in the prediction 

15 error coding block in an efficient form for transmission, for example 
using a discrete cosine transform (DCT). The coded prediction error 
block is sent to multiplexer 13 for further transmission to the decoder. 

The encoder of the digital image transmission system also includes 

20 decoding functionality. The encoded prediction error of the current 
block is decoded in prediction error decoding block 30 and is 
subsequently summed in summer 31 with the predicted pixel values for 
the current block. In this way, a decoded version of the current block is 
obtained. The decoded current block is then directed to a block 

25 boundary filter 32, implemented according to the method of the 
invention. Now referring to the flow chart of Figure 6, the block 
boundary filter 32 examines 602, 604, if the current (just decoded) 
block has boundaries that can be filtered. In order to determine if such 
boundaries exist, the block boundary filter examines the contents of the 

30 frame buffer 33. If more than one such boundary exists, the block 
boundary filter determines 603 the filtering order to be used in filtering 
the boundaries. If at least one boundary is found, the block boundary 
filter retrieves 605 those pixel values that belong to the adjacent block 
of the present boundary to be used in the filtering process. The block 

35 boundary filter performs the filtering 606 according to a preferred 
filtering method and updates at least the modified pixel values in the 
current block and the values of pixels filtered in previously decoded 
blocks stored in the frame buffer 33. The block boundary filter then 
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examines 608 whether there are still boundaries to be filtered. If other 
boundaries are to be filtered the process returns to step 605. The block 
filter performs analogous operations on each block of the frame being 
coded until all blocks have been encoded. As each block is filtered it is 
5 made available for use e.g. in the prediction and/or filtering subsequent 
blocks by storing it in the frame buffer 33. 

The operation of the block boundary filtering method according to the 
invention will now be described in connection with the receiver of a 
10 digital image transmission system. Here, use of the filter is described in 
connection with the decoding of a frame, assumed to have been 
encoded in Intra format using some form of Intra prediction method in 
which the pixel values of a current block are predicted on the basis of 
previously encoded image blocks within the same frame. 

15 

Here it is also assumed that the receiver receives the blocks that form a 
digital image frame one by one from a transmission channel. It should 
be appreciated that in other embodiments, the receiver may receive a 
complete frame to be decoded, or alternatively may retrieve the digital 
20 image to be decoded from a file present on some form of storage 
medium. In any case, operation of the block boundary filtering method 
according to the invention is performed on a block-by-block basis, as 
described below. 

25 In the receiver 60, a demultiplexer receives and demultiplexes coded 
prediction error blocks and prediction information transmitted from the 
encoder 50. Depending on the prediction method in question, the 
prediction information may include parameters used in the prediction 
process. It should be appreciated that in the case that only one Intra 

30 prediction method is used, information concerning the prediction 
method used to code the blocks is unnecessary, although it may still be 
necessary to transmit parameters used in the prediction process. In 
Figure 5, dotted lines are used to to represent the optional transmission 
and reception of prediction method information and/or prediction 

35 parameters. Assuming more than one Intra prediction method may be 
used, information concerning the choice of prediction method for the 
current block being decoded is provided to Intra prediction block 41 . 
Intra prediction block 41 examines the contents of frame buffer 39 to 
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determine if there exist previously decoded blocks to be used in the 
prediction of the pixel values of the current block. If such image blocks 
exist, Intra prediction block 41 predicts the contents of the current block 
using the prediction method indicated by the received prediction 
5 method information and possible prediction-related parameters 
received from the encoder. Prediction error information associated with 
the current block is received by prediction error decoding block 36 
which decodes the prediction error block using an appropriate method. 
For example, if the prediction error information was encoded using a 

10 discrete cosine transform, the prediction error decoding block performs 
an inverse DCT to retrieve the error information. The prediction error 
information is then summed with the prediction for the current image 
block in summer 37 and the output of the summer is applied to block 
boundary filter 38. Block boundary filter 38 applies boundary filtering to 

15 the newly decoded image block in a manner analogous to the block 
boundary filter of the encoder (block 32). Accordingly, block boundary 
filter 38 examines 602, 604, if the current (newly decoded) block has 
boundaries that can be filtered. In order to determine if such boundaries 
exist, the block boundary filter 38 examines the contents of frame buffer 

20 39 which contains previously decoded and reconstructed image blocks. 
If more than one such boundary exists, the block boundary filter 
determines 603 the filtering order to be used in filtering the boundaries. 
Advantageously, this order is identical to that used in the boundary filter 
32 of the encoder. If at least one boundary is found, the block boundary 

25 filter retrieves 605 those pixel values that belong to the adjacent block 
of the present boundary for use in the filtering process. The block 
boundary filter performs the filtering 606 according to a preferred 
filtering method (advantageously identical to that used in the encoder) 
and updates at least the modified pixel values in the current block and 

30 the values of pixels filtered in previously decoded blocks stored in the 
frame buffer 39. The block boundary filter then examines 608 whether 
there are still boundaries to be filtered. If other boundaries are to be 
filtered the process returns to step 605. 

35 The block filter performs analogous operations on each block of the 
frame, substantially immediately after each block is decoded, until all 
blocks have been decoded and their boundaries appropriately filtered. 
As each block is filtered it is made available for use e.g. in the 
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prediction and/or filtering subsequent blocks by storing it in the frame 
buffer 39. Furthermore, as each block is decoded and its boundaries 
filtered by applying the method according to the invention, it is directed 
to the output of the decoder 40, for example to be displayed on some 
5 form of display means. Alternatively, the image frame may be displayed 
only after the whole frame has been decoded and accumulated in the 
frame buffer 39. 

In the paragraphs above, the method according to the invention was 

1 0 described in connection with the filtering of block boundaries in a frame 
coded in Intra format and further using Intra prediction methods. It 
should be appreciated that the method according to the invention may 
be applied in an exactly analogous manner for filtering block 
boundaries between Intra coded blocks that form part of an otherwise 

15 Inter-coded frame. Alternatively, the method may be applied to Inter 
coded image blocks. In this case, each Inter coded block is predicted 
based on information concerning its motion between a current frame 
and a reference frame and the operations of the encoder are as 
follows: A prediction error block is formed based on the difference 

20 between the prediction for the block and the actual contents of the 
block. The prediction error block is coded in a way known as such, for 
example using a DCT and transmitted to the decoder together with 
information, for example motion coefficients, representing the motion of 
the block. In the encoder, the coded prediction error is further decoded 

25 and summed with the prediction for the current block to produce a 
decoded and reconstructed block that forms part of a prediction 
reference frame to be used in connection with motion compensated 
prediction coding of the next frame in the sequence. As was the case in 
the example given above concerning Intra coding, a block boundary 

30 filter can be implemented for use in connection with Inter coded blocks 
in such a way that it receives and filters Inter coded blocks of the 
current frame substantially immediately after they have been decoded. 
An equivalent arrangement may be used in connection with the 
decoding of Inter coded blocks received at the receiver of a digital 

35 image transmission system. 

The block carrying out the filtering method according to the invention is 
particularly advantageously implemented in a digital signal processor or 



24/01 '00 19;08 FAX 03 2886262 



TAMPEREEN PATENT 



@020 



18 



a corresponding general purpose device suited to processing digital 
signals, which can be programmed to apply predetermined processing 
functions to signals received as input data. The measures according to 
Figure 6 can be carried out in a separate signal processor or they can 
5 be part of the operation of such a signal processor which also contains 
other arrangements for signal processing. 

A storage medium can be used for storing a software program 
comprising machine executable steps for performing the method 
10 according to the invention. Then, in an advantageous embodiment of 
the invention, the software program can be read from the storage 
medium to a device comprising programmable means, e.g. a processor, 
for performing the method of the invention. 

15 In the method and device according to the invention, the number of 
pixels to be selected for filtering can vary, and it is not necessarily the 
same on different sides of the block boundary. The number of pixels 
may be adapted according to the general features of the image inform 
mation contained by the frame. Furthermore, many filtering methods 

20 can be applied with the present invention. In some intra-prediction 
methods it is not necessary to send the intra prediction information in 
addition to the coded differential blocks to the receiver 60. The 
definitions of filtering order above have also been intended as 
examples only. 

25 

A particularly advantageous use of the invention is in mobile 
teleconferencing applications, digital television receivers and other 
devices that at least receive and decode digital video images. 

30 Figure 7 presents a simplified schematic diagram of a mobile terminal 
41 intended for use as a portable teleconferencing device and applying 
the deblocking filter method according to the invention. The mobile 
terminal comprises advantageously at least display means 42 for dis- 
playing images, audio means 43 for audio information, keyboard 44 for 

35 inputting e.g. user commands, radio part 45 for communicating with a 
mobile telecommunications network, processing means 46 for 
controlling the operation of the device, memory means 47 for storing 
information, and preferably a camera 48 for taking images. 
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The present invention is not solely restricted to the above presented 
embodiments, but it can be modified within the scope of the appended 
claims. 
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Claims: 

1 . A method for removing visual artefacts from a frame of a digital 
image, which has been coded by blocks (B1, B2, B3, B4) and then 

5 decoded, in which method at least one pixel from at least one side of a 
block boundary (R12, R13, R24, R34) is selected for filtering, filtering is 
performed on the block boundary (R12, R13, R24, R34) between two 
adjacent blocks (B1, B2, B3, B4), and the blocks are scanned in a 
certain order, characterized in that filtering on the block boundary 

10 (R12, R13, R24, R34) is performed substantially immediately after the 
block (B1 , B2, B3, B4) is decoded, and that the order of filtering the 
boundaries is predetermined. 

2. A method according to Claim 1 , characterized in that after a block 
(B1 , B2, B3, B4) is decoded it is examined (603, 604) to determine, if 

15 there exists another decoded block (B1, B2, B3, B4) adjacent to said 
decoded block under examination, wherein if such a block is found, 
filtering is performed on the boundary (R12, R13, R24, R34) between 
said decoded block under examination and said other decoded block. 

3. A method according to Claim 2, characterized in that if there 
20 exists more than one decoded block (B1 , B2, B3, B4) adjacent to said 

decoded block under examination, filtering is performed in a certain 
order on the boundaries (R12, R13, R24, R34) between said decoded 
block under examination and said other decoded blocks. 

4. A method according to Claim 2 or 3, characterized in that a value 
25 of at least one pixel selected for examination is corrected by filtering, 

and that at least one corrected value of filtered pixels is used in filtering 
at least on one other block boundary (R12, R13, R24, R34). 

5. A method according to Claim 2, 3 or 4, characterized in that intra 
prediction of a subsequent block is performed after the block (B1, B2, 

30 B3, B4) is decoded, that a value of at least one pixel selected for 
examination is corrected by filtering, and that at least one corrected 
value of filtered pixels is used in the intra prediction of at least one 
subsequent block. 
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6. A method according to Claim 2, 3, 4 or 5, characterized in that the 
blocks of the frame are grouped into macroblocks, wherein the frame is 
scanned macroblock by macroblock. 

7. A method according to Claim 6, characterized in that the frame is 
5 scanned horizontally from top-left to bottom-right. 

8. A method according to Claim 7, characterized in that in a situation 
where at least two boundaries are filtered after the block (B4) is 
decoded, the filtering order is selected such that the left boundary 
(R34) of said block is filtered before the top boundary (R24). 

10 9. A device for removing visual artefacts from a frame of a digital 
image, which has been coded by blocks (B1, B2, B3, B4) and then 
decoded, which device comprises means for selecting at least one pixel 
from at least one side of a block boundary (30) for filtering, means for 
performing filtering on a block boundary (R12, R13, R24, R34) between 

15 two adjacent blocks (B1, B2, B3, B4), and means for scanning the 
blocks in a certain order, characterized in that filtering on the block 
boundary (R12, R13, R24, R34) is performed substantially immediately 
after the block (B1, B2, B3, B4) is decoded, and that the order of 
filtering the boundaries is predetermined. 

20 10. A device according to Claim 9, characterized in that it comprises 
means for determining, if there exists another decoded block (B1, B2, 
B3, B4) adjacent to said decoded block under examination, wherein if 
• :• such a block is found, filtering is performed on the boundary (R12. R13, 

R24, R34) between said decoded block under examination and said 

25 other decoded block. 

* 11. A device according to Claim 10, characterized in that it comprises 

:[[[: means for examining, if there exists more than one decoded block (B1, 

B2, B3, B4) adjacent to said decoded block under examination, wherein 
the means for performing filtering on a block boundary (R12, R13, R24, 
\ll m 30 R34) comprises means for performing the filtering in a certain order on 
the boundaries (R12, R13, R24, R34) between said decoded block 
*•***• under examination and said other decoded blocks. 

* « * 

12. A device according to Claim 10 or 11, characterized in that it 
. ,;i a comprises a filter for correcting a value of at least one pixel selected for 

« * 

t « t 
t • • 
« t 
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examination, means for saving the results of filtering on a block 
bpundary (R12, R13, R24, R34), and means for using the saved results 
in filtering at least on one other block boundary (R12, R13, R24, R34). 

13. A device according to Claim 10, 11 or 12, characterized in that it 
5 comprises means for performing intra prediction of a subsequent block 

after the block (B1 , B2, B3, B4) is decoded, a filter for correcting a 
value of at least one pixel selected for examination, means for saving 
the results of filtering on a block boundary (R12, R13, R24, R34), and 
means for using the saved results in intra prediction of at least on one 
10 other block boundary (R12, R13, R24, R34). 

14. A device according to any of the Claims 1 0 to 13, characterized in 
that the blocks of the frame are grouped into macroblocks, wherein the 
frame is arranged to be scanned macroblock by macroblock. 

15. A device according to Claim 14, characterized in that the frame is 
1 5 scanned horizontally from top-left to bottom-right. 

16. A device according to Claim 15, characterized in that the filtering 
order is selected such that the left boundary (R34) of said block is 
filtered before the top boundary (R24). 

17. An encoder (50) comprising means for coding and decoding a 
20 digital image by blocks (B1, B2, B3, B4), which encoder comprises 

means for selecting at least one pixel from at least one side of a block 
boundary (30) for filtering, means for performing filtering on a block 
boundary (R12, R13, R24, R34) between two adjacent blocks (B1, B2, 
B3, B4), and means for scanning the blocks in a certain order, 
25 characterized in that filtering on the block boundary (R12, R13, R24, 
R34) is performed substantially immediately after the block (B1, B2, B3, 
B4) is decoded, and that the order of filtering the boundaries is 
predetermined. 

18. A decoder (60) comprising means for decoding a digital image by 
30 blocks (B1 , B2, B3, B4), which decoder comprises means for selecting 

at least one pixel from at least one side of a block boundary (30) for 
filtering, means for performing filtering on a block boundary (R12, R13, 
R24, R34) between two adjacent blocks (B1, B2. B3, B4), and means 
for scanning the blocks in a certain order, characterized in that filtering 
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on the block boundary (R12, R13, R24, R34) is performed substantially 
immediately after the block (B1, B2, B3, B4) is decoded, and that the 
order of filtering the boundaries is predetermined. 

19. A codec (50, 60) comprising means for coding and decoding a 
5 digital image by blocks (B1 , B2, B3, B4), which codec comprises means 

for selecting at least one pixel from at least one side of a block 
boundary (30) for filtering, means for performing filtering on a block 
boundary (R12, R13, R24, R34) between two adjacent blocks (B1, B2, 
B3, B4), and means for scanning the blocks in a certain order, 
10 characterized in that filtering on the block boundary (R12, R13, R24, 
R34) is performed substantially immediately after the block (B1 , B2, B3, 
B4) is decoded, and that the order of filtering the boundaries is 
predetermined. 

20. A mobile terminal comprising a video codec (50, 60), which 
15 comprises means for coding and decoding a digital image by blocks 

(B1 , B2, B3, B4), which codec comprises means for selecting at least 
one pixel from at least one side of a block boundary (30) for filtering, 
means for performing filtering on a block boundary (R12, R13, R24, 
R34) between two adjacent blocks (B1, B2, B3, B4), and means for 
20 scanning the blocks in a certain order, characterized in that filtering on 
the block boundary (R12, R13, R24, R34) is performed substantially 
immediately after the block (B1, B2, B3, B4) is decoded, and that the 
order of filtering the boundaries is predetermined. 

21. A storage medium for storing a software program comprising 
25 machine executable steps for coding and decoding a digital video 

signal by blocks, for selecting at least one pixel from at least one side of 
a block boundary (30) for filtering, for performing filtering on a block 
boundary (R12, R13, R24, R34) between two adjacent blocks (B1, B2, 
B3, B4), and for scanning the blocks in a certain order, characterized 
30 in that the software program further comprises machine executable 
steps for performing filtering on the block boundary (R12, R13, R24, 
R34) substantially immediately after the block (B1, B2, B3, B4) is 
decoded, and that the order of filtering the boundaries is 
< predetermined. 



24/01 00 10:09 FAX 03 2886262 



TAMPEREEN PATENT 



Abstract: 

The invention relates to a method for removing blocking 
and other blocking related visual artefacts from the 
frame of a digital image, which has been coded by 
blocks (B1, B2, B3, B4) and then decoded. In the 
method at least one pixel from at least one side of a 
block boundary (R12, R13, R24, R34) is selected for 
filtering. Filtering is performed on the block boundary 
(R12, R13, R24, R34) between two adjacent blocks (B1, 
B2, B3, B4). The blocks are scanned in a certain order. 
Filtering on the block boundary (R12, R13, R24, R34) is 
performed substantially immediately after the block (B1 , 
B2, B3, B4) is decoded, and the order of filtering the 
boundaries is predetermined. 



Fig. 4 
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